Generating composite or panoramic images, or more simply panoramas, from sets of still images or sequences of video frames (collectively “frames”) is known. In this manner, information relating to the same physical scene at a plurality of different time instants, viewpoints, fields of view, resolutions, and the like from the frames is melded to form a single wider angle image.
To generate a panorama, the various frames are geometrically and calorimetrically registered, aligned and then merged or stitched together to form a view of the scene as a single coherent image. During registration, each frame is analyzed to determine if it can be matched with previous frames. A displacement field that represents the offset between the frames is determined and then the frame is warped to the others to remove or minimize the offset.
In order for the panorama to be coherent, points in the panorama must be in one-to-one correspondence with points in the scene. Accordingly, given a reference coordinate system on a surface to which the frames are warped and combined, it is necessary to determine the exact spatial mapping between points in the reference coordinate system and pixels of each frame.
For any given panorama, the individual frames that are used to form the panorama may have been captured under very different conditions i.e. at different times of the day, different cloud coverage etc. and thus, may have very different brightness levels. As such noticeable differences in brightness may appear in the panorama at the regions of frame stitching. Techniques to reduce brightness inconsistencies in adjoining frames have been considered.
For example, U.S. Pat. No. 6,249,616 to Hashimoto discloses a method for making transformation in brightness globally consistent by treating transformation parameters computed on a local basis as estimates for the global parameters. A covariance matrix is then calculated to provide relative confidence in the estimate of each of the transformation parameters.
U.S. Pat. Nos. 6,385,349 and 6,128,108, to Teo disclose a method for stitching together images, which minimizes the “zone of influence” that adjacent images have on one another. Initially, two adjacent images A and B are merged into a combined image AB, and then the combined image AB is merged with a third image C to form a longer combined image ABC. This process is continued for each remaining image. The influence of each merged image is limited to the width of a single image.
Other techniques for reducing brightness variations in panoramas have also been considered. For example, U.S. Pat. No. 6,385,349 to Teo discloses a method for compositing images, which calculates modified intensities of individual images by using a calculated gamma as a modifying power on the intensity of the images. Correction parameters between adjacent images are determined in order to minimize the effect of lighting differences between the images.
U.S. Patent Application Publication No. 2004/0061774 to Wachtel et al. discloses a system and method for forming seamless composite images. The system comprises a number of imaging devices that are centrally controlled to capture respective images. A mean brightness is calculated from a center image, and is used as a base line for brightness coordination amongst the other images. As such, the intensities of the other images are adjusted based on the mean brightness.
U.S. Patent Application Publication No. 2004/0008267 to Chen et al. discloses a method for providing an image having an extended dynamic range whereby a number of limited dynamic range images of a particular scene are captured and combined.
U.S. Pat. No. 6,754,379 to Xiong et al. discloses a user interface for enabling a user to adjust image parameters such that constituent images in a composite image may be combined seamlessly.
U.S. Pat. No. 5,828,793 to Mann also discloses a method and apparatus for producing extended dynamic range digital images. Component images of the same scene taken at different exposures are received and combined to form the extended dynamic range image.
Although the above references disclose techniques for reducing brightness variations in a panorama, improvements are desired. It is therefore an object of the present invention to provide a novel method, apparatus and computer program for reducing brightness variations in a panorama.